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Executive Summary 



THE National JTPA Study was commissioned by the Employment and Training 
Administration of the U.S. Department of Labor (DOL) in 1986 to measure the 
impacts and costs of selected employment and training programs funded under Title II-A 
of the Job Training Partnership Act of 1982, which is targeted to serve economically 
disadvantaged Americans. This Executive Summary presents estimates of the impacts of 
the program on the earnings and employment of adults and out-of-school youths in 16 local 
service delivery areas during the first 18 months after acceptance into the program. 

The report on which this Executive Summary is based presents interim findings of the 
study (Bloom, On, Cave, Bell, and Dooliule, forthcoming). Estimates of longer term 
program impacts on earning?, employment, and welfare benefits, and an analysis of 
program costs and benefits, will appear in the final report of the study (forthcoming, from 
Abt Associates Inc.). A companion report on the study's implementation (Kemple, 
Doo little, and Wallace, forthcoming), describes the JTPA programs operated in the study 
sites and the types of JTPA-funded services provided to members of the study sample. 



The National JTPA Study 

This study grew out of the recommendations of the Job Training Longitudinal Study 
Advisory Panel, a group of nationally recognized experts in employment and training 
research formed to advise DOL on the evaluation of JTPA (Stromsdorfer et al., 1985). 
After reviewing evaluations of the Comprehensive Employment and Training Act (CETA) 
programs, the Panel concluded mat the only reliable way to measure the impacts of 
employment and training programs was to conduct a classical experiment, in which 
program applicants are randomly assigned either to a treatment group, which is allowed 
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access to the program, or to a control group, which is not. Random assignment assures 
that the two groups do not differ systematically in any way except access to the program. 
Thus, subject only to the uncertainty associated with sampling error, any subsequent 
differences in outcomes between the two groups can confidently be attributed to the 
program. These differences are termed the program impacts. 

Although random assignment designs have been used to evaluate a number of 
demonstration projects and state programs, the Employment and Training Administration 
was the first federal agency to apply this approach to an ongoing national program. Because 
of its rigorous design, the National JTPA Study provides the first fully reliable estimates 
of the impacts of the largest employment and training program sponsored by the federal 
government. 

In the National JTPA Study 20,601 JTPA applicants in 16 service delivery areas (SDAs) 
across the country were randomly assigned to the treatment group or the control group over 
the period November 1987 through September 1989. The earnings and employment 
outcomes of both groups were then measured through follow-up surveys and administrative 
records obtained from state unemployment insurance agencies. Data on the baseline 
characteristics of the two groups were collected as part of the program intake process, and 
information about the employment and training services received was obtained from 
follow-up surveys and SDA records. 

The study sites were not chosen to be representative of the nation in a statistical sense, 
but they do reflect the diversity of local programs and local environments in JTPA. 1 In 
particular, the performance of the sites during the study period, as measured by JTPA 
performance indicators, was not noticeably different from that of all SDAs nationally. 



The 18-Month Impact Analysis 

This report provides estimates of the impact of JTPA Title II-A on the earnings and 
employment of four target groups— adult women and men (ages 22 and older) and female 
and male out-of-school youths (ages 16 to 21)— over the first 18 months after random 
assignment. Adult women make up 30 percent of the national JTPA population; adult men, 



1. See Bloom, On, Cave, Bell, and Doohttle (forthcoming) and Kemple, Doolirtk, and Wallace 
(forthcoming) for comparisons of the 16 study sites with all SDAs nationally. 
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25 percent; and out-of-school youths, 23 percent. In-school youths, who are not inclvded 
in this study, form the remaining 22 percent. 

The analysis is based on a sample of 17,026 sample members whose First Follow-up 
Survey interview was scheduled at least 18 months after random assignment. 2 For each 
target group, we estimated impacts for a number of different subgroups, defined by the 
types of program services recommended for them and by their baseline characteristics. 

Because the study was designed to measure the effects of JTPA as it normally operates, 
the analysis investigates which JTPA-funded services were working well for those 
recommended to receive them; the analysis does not assess possible alternatives to the 
existing program. By identifying those groups for whom Title H-A is having positive 
effects and those for whom it is having no effect— or even a nwjative effect— we hope to 
help policymakers in their efforts to identify and improve those parts of the program that 
need improvement. This analysis cannot, however, tell policymakers tow to improve the 
program, since it does not compare alternative programs for similar people. Rather, it 
measures only the effects of the existing program on the people it actually served over the 
study period. 

In the remainder of this Executive Summary we first provide an overview of the 
estimated effects of the program on the earnings and employment of the four main target 
groups— adult women ami men, and female and male youths. We then present more detailed 
findings for adult and youth subgroups in turn, and conclude with implications of the 
findings for the JTPA program and future research. 

Overall Impacts on Earnings and Employment, by Target Group 

JTPA Title n-A had generally positive effects on the earnings and employment of adults 
in the study sites. As shown in the top panel of Exhibit S.l, access to the program increased 
the average 18-month earnings of the adult women randomly assigned to the treatment 
group ("JTPA assignees") by an estimated $539, or 7 .2 percent of the control group mean. 
Access to the program also increased the percentage of women employed at some time 
during the follow-up period by 2.1 percentage points. Because these estimates are 



2 . Within this 1 8-month study ample, First Follow-up Survey data are available for 1 4,446 sample 
members, or 84.8 percent of the sample. 
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Exhibit S. I Impacts on Total 18-Month Earnings and Employment: J TP A Assignees 
and Enrvllees. by Target Croup 



Adults Out-of-school youths 





women 


Men 


tenuue 


Male 


Impact on: 


(I) 


(2) 


(3) 


(4) 






Per assignee 




Earnings 
In$ 


$ 539*** 


$ 550 


$ -182 


$ -854** 


Asa % 




45% 


-2.9% 


-7.9% 


Percentage employed" 


2.1** 


2.8** 


2.8 


15 


Sample size (assignees 
and control group) 


6,474 


4.419 


2JOQ 


1,748 






Perenrollee 




Earnings 
In$ 


$ 873* 


$ 935* 


% -295* 


$-1,355* 


As a % 


122% 


6.8% 


-4.6% 


-11.6% 


Percentage employed" 


3.5* 


4.8* 


4.5* 


2.4* 



e. Ai eny time during the lailow-up period .^^^^^ — — 
». Tesu of BsdBic&l ngmflccsoc were «* performed for impact* per eruoUe*. 
• SutiitkaUy ngmficus it itao .10 level, at the .05 level, •** S ibe .01 level (two-uiled tea). 



statistically significant (as indicated by the asterisks beside them), we take them to be 
reliable evidence of positive impacts on earnings. In this analysis we accept only statisti- 
cally significant estimates as evidence of real program effects. 

The estimated program impacts on the earnings of adult men— an earnings gain of VJ50, 
or 4.5 percent, and an increase in the percentage employed of 2.8 percentage points— were 
similar in size to those for adult women, but the estimated impact on earnings was not 
statistically significant. 

In contrast to the findings for adults, the program had little or no effect on the average 
earnings of female youths (a statistically insignificant earnings loss of -5182, or -2.9 
percent), and the program actually reduced the earnings of male youths— as evidenced by 
a large, statistically significant loss of -$854, or -7.9 percent, over the 18-month period. 
Access to JTPA had no significant effect on the employment rates of either female or male 
yot'Jis. 

The estimates discussed above are average impacts on the earnings and employment of 
all sample members assigned to the treatment group. Although all of these assignees were 
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given access to JTPA, not all of them actually enrolled in the program. The bottom panel 
of Exhibit S.l presents our best estimate of program impacts on the earnings and 
employment of JTPA enrollees (assignees who were later enrolled). 3 

Estimated impacts per enrollee— both gains and losses— were about 60 percent to 70 
percent larger than impacts per assignee, depending on the target group. The estimated 
earnings gains of adult women and men who were enrolled in JTPA were $873 and $935, 
respectively. Impacts on youths were earnings losses of -$295 for females and -$ 1 ,355 for 
males. The impact of the program on the percentage of enrollees in each target group who 
were employed ranged from an increase of 2.4 percentage points for male youths to an 
increase of 4.8 percentage points for adult men. 

It is important to understand that the impact per assignee and the impact per enrollee are 
not two different estimates of the overall effect of the program. They simply spread the 
total estimated program effect on the sample over a larger group (assignees) or a smaller 
group (enrollees). Thus, the two sets of estimates are entirely consistent; they just measure 
different concepts. In the remainder of this Executive Summary, we focus on the estimated 
impacts per a^ignee, because they are the most reliable, direct experimental evidence of 
the effects of the program. 

Impacts on earnings reflect program effects on both the amount of time treatment group 
members worked and how much they were paid per hour worked. Exhibit S.2 shows 
estimated impacts on the average number of hours worked and average earnings per hour 
worked over the follow-up period, expressed as percentages of the corresponding control 
group means. The percentage impacts on these two components of earnings approximately 
sum to the percentage impact on total earnings per assignee/ 



3. To derive estimates for enrollees, it was necessary to assume that there was no impact on the 
earnings and employment of nonenrollees. There is evidence, however, that about half of all 
iiuucurcL'fss had some contact with the program after random assignment and received some— 
usually minimal- -program services. As a result, the estimates in the bottom panel probably overstate 
somewhat the true impact per enrollee, while the estimated impacts per assignee understate the true 
impact per enrollee. Thus, the true impact on enrollees probably lies somewhere between these two 
estimates. The estimates for enrollees sisa adjust for the fact that 3 percent of the control group were 
enrolled in JTPA, despite the experiment's embargo on their participation. 

4. Because the impacts on earnings per hour worked were estimated indirectly, we did not 
calculate significance levels for these impacts. 
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Exhibit 5.2 Percentage Impacts on Total 18-Month Earnings ami Its Components: 
JTPA Assignees and Enrollees, by Target Croup 





Adult 


Adult 


Female 


Male 


Percentage 


women 


men 


youths 


youths 


impact an: 


w 


a> 


(3) 


(4) 


Earnings per assignee 


72%*** 


4.5% 


-2.9% 


-7.9%** 


Hours worked per assignee 


3.7 


4.5** 


-4.7 


-6.8** 


Earnings per hour worked 


3.4" 


0.0" 


1.8" 


At 


Sample size 


6,474 


4.419 


2J00 


1,748 



a. Tesu of itatuticaJ Hgxs fierce were not performed for input* on earning* per hour worked. 
• SutuAkail? tigrufkenS u the .10 level, •"• at ifae .05 level, «1 0» .01 level (two-tailed ten). 



As shown in the exhibit, the 7.2 percent increase in earnings for adult women reflects 
a combination of a 3.7 percent increase in hours worked and a 3.4 percent increase in 
average hourly earnings among those who worked. The earnings gain for adult men, on 
the other hand, was entirely attributable to a 4.5 percent rise in hours worked, with no 
increase in hourly earnings. 

Among female youths a -4.7 percent reduction in hours worked more than offset a 1 .8 
percent increase in hourly earnings to produce the negligible impact on total earnings that 
we saw earlier. Among male youth the -7.9 percent loss in total earnings was primarily 
attributable to a decrease in hours worked (of -6.8 percent). 

Overall, then, JTPA appears to have had modest positive effects on the earnings and 
employment of adult women and men, little or no effect on the earnings and employment 
of female youths, and substantial negative effects on the earnings and employment of male 
youths. 

Findings for Subgroups of Adult Women and Men 

The impacts presented in the previous section are estimates of the average effect of the 
program on each target group in the study sample. Because JTPA provides a number of 
different employment and training services to a wide range of program applicants, it is 
important to analyze how program impacts varied with the type of service offered and the 
characteristics of the applicants. In this section we therefore present estimates of program 
impacts on the earnings of subgroups of adults, defined by the services program intake staff 
recommended for them and by selected personal characteristics. 



If; 
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Service Strategies Recommended 

For purposes of this analysis, members of the study sample were classified into three service 
strategy subgroups ased on the services program intake staff recommended for each 
sample member prior to random assignment. 5 Applicants recommended for classroom 
training in occupational skills were placed in the classroom training subgroup. Those 
recommended for on-the-job training (OJT) were placed in the OJT/JSA subgroup (so 
named because many of the treatment group members in this subgroup were enrolled in job 
search assistance while searching for either an on-the-job training position or an 
unsubsidized job). Because JTPA staff often recommend combinations and sequences of 
services, applicants placed in either of these subgroups may also have been recommended 
for any of several other services, including job search assistance, basic education, work 
experience, or miscellaneous other services. Those applicants recommended for one or 
more of these services— but neither classroom training in occupational skills nor on-the- 
job training—were placed in the third subgroup: other services.* 

As shown in Exhibit S.3, nearly half of all adult men in the sample were recommended 
for the OJT/JSA service strategy, with the remainder about equally divided between the 
classroom training and other services strategies. Women were more likely than men to be 
recommended for classroom training (44 percent versus 25 percent) and less likely to be 
recommended for OJT/JSA (35 percent versus 49 percent). 

It is important to note that program intake staff made their recommendations based on 
the employment needs and qualifications of the individual applicants, as well as their 
personal preferences. The service strategy subgroups therefore differed from one another 
not only in terms of the service recommendations but also in terms of personal 
characteristics. 



5. Service strategy subgroups *ere defined based on the services recommended rather than the 
services received for two reasons. First it was not possible to identify control group members who 
were comparable to the treatment grcup members who received particular JTPA services, whereas 
it y,as possible to identify control group members who were recommended for the same services as 
treatment group members. Second, and more fundamentally, since program staff can recommend 
services but cannot ensure that applicants participate in those services, recommended services 
represent the operative program decision to be evaluated. 

6. A few applicants designated for this other service subgroup were recommended for classroom 
training in occupational skills or on-the-job training as part of "customized training." 
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Exhibit SJ Service Strategies Recommended: 
Adult JTPA Assignees, by Gender 





Adult 


Adult 




women 


men 


Service strategy 


(I) 


(2) 


Classroom training 


44.0% 


24.6% 


OJT/JSA 


35.0 


48.7 


Other services 


21.0 


26.7 


Sample size 


4,465 


3.759 



Enrollment Rates and Duration, by Service Strategy Subgroup 

After assessment and recommendation of services, two-thirds of the applicants accepted by 
intake staff were randomly assigned to the treatment group, which was allowed access to 
JTPA, and one-third were assigned to the control group, which was excluded from JTPA 
for 18 months. 7 

As noted above, not all treatment group members would ultimately become enrolled in 
JTPA. Enrollment rates differed by service strategy subgroup, but overall they were quite 
similar for adult women and men. Within the treatment group as a whole, 65 percent of 
adult women and 61 percent of adult men were enrolled in JTPA at some time during the 
1 8-month follow-up period. Enrollment rams were highest in the classroom training 
subgroup (73 percent and 71 percent for adult women and men, respectively) and lowest 
- the OJT/JSA subgroup (55 percent and 57 percent, respectively). 

The duration of enrollment in the program also differed by service strategy, ranging 
from a median length of about 2 months for women and men in the OJT/JSA and other 
services subgroups to median lengths of enrollment of 4 to 6 months in the classroom 
training iubgroup. There was little difference by gender in the duration of enrollment 
except that women in classroom training tended to stay in the program about two months 
longer man men. 



7. This embargo on services to control group members was successfully implemented. Over the 
course of the 18-month follow-up period, only 3 percent of coi sol group members became enrolled 
in JTPA. 
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Services Received, by Service Strategy Subgroup 

Within the classroom training subgroup the most common JTPA services received by 
treatment group members who became enrolled in the program were classroom training in 
occupational skills and basic education. Enroilees in the OJT/JSA subgroup were most 
likely to receive on-the-job training or job search assistance, or both. In the other services 
subgroup the most common services adults received were job search assistance and 
miscellaneous services, such as job-readiness training. Exhibit S .4 show that between 82 
percent and 89 percent of the enroilees in each service strategy subgroup received one or 
both of the two most common services received by members of that subgroup. Thus, the 
three service strategy definitions represent distinctly different mixes of services actually 
received, as well as services recommended. 

The impacts of the program do not depend solely, however, on the JTPA services 
received by those in the treatment group . Instead, the impacts reflect the difference between 
the services received by those given access to JTPA and the services they would have 
received if they had been excluded from the program. That is, the benchmark against which 
we measure the effects of JTPA is the services available and used elsewhere in the 
community, not a total absence of services. Our measure of the services the treatment group 
would have received if they had been excluded from th* program is those received by the 
control group, who were excluded from the program. 

Since we measure impacts per assignee (treatment group member), the relevant 
comparison is in terms of services per assignee, including those who were never enrolled 
in JTPA. As expected, the largest treatment-control group difference in the classroom 
training subgroup was In receipt of classroom training in occupational skills. Among adult 
women 49 percent of the treatment group received this service, whereas only 29 percent 
of the control group did. Among adult men these figures were 40 percent versus 24 percent. 

Aault treatment group members in the OJT/JSA subgroup were much more likely than 
control group nr abers to receive on-the-job training. Twenty-nine percent of the women 
and 27 percent * the men in the treatment group in this subgroup received OJT, whereas 
less than 1 pertwut of the corresponding control groups received this service, since OJT 
is typically not funded by non-JTPA providers. We were not able to measure control group 
receipt of job search assistance from non-JTPA providers, and so we could not estimate 
the treatment-control group difference for that service. 
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Exhibit S. 4 Most Common JTPA Services Received by Treatment Group 
Members Who Were Enrolled in the Program: Adults, by 
Gender and Service Strategy Subgroup 

% ofenrollees receiving 
one or both services 



A~duB Adult 
Two most common services women men 
in service strategy subgroup (J) (V 





Classroom training subgroup 


Classroom training in occupational 




skills/basic education" 


88.8% 85.5% 




OJTUSA subgroup 


On-the-job training/ 




job search assistance 


87.8% * 865% 




Other services subgroup 


Job search assistance/ 




miscellaneous * 


823% 88.7% 


Sample size 


2JS95 2J05 



a. 'Banc education" includes Adult Bute Education (ABE), tegh school orGeaenl Educational 

Development (GEO) preparation, sad Bnglisc ss s Second Langusge (ESL). 
». "Miscellaneous" includes sasesssKSJt,job-eeadlacss uaimng, customised training, vocations! exploration 
job ihsdowiog, sad tiyout emnlovmenl. smoag other services. 



As noted earlier, the most common JTPA services provided to adults in the other services 
subgroup were job search assistance and miscellaneous services. Around 25 percent of 
adult treatment group members in this subgroup received the former service, and around 
30 percent received the latter. We were unable to measure receipt of these services from 
non-JTPA providers and therefore cannot estimate the treatment-control group difference. 

On the basis of these data, we conclude that JTPA represented a substantial increment 
to the services that were available at the 16 study sites, at least in the cases of the classroom 
training and OJT/JSA subgroups, for which we could measure the increment. 

Impacts on Educational Attainment, by Service Strategy Subgroup 

An intermediate effect of the incremental services received by the treatment group was an 
increase in educational attainment among those high school dropouts who received some 
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employment or training service during the 18-month follow-up period. Dropouts made up 
around 30 percent of the adult target groups. 

As might be expected, the increase in educational attainment was greatest among those 
dropouts recommended for the classroom training service strategy. Exhibit S .5 indicates 
that nearly 30 percent of the adult dropouts in the classroom training treatment group 
received a training-related high school diploma or General Educational Development 
(GED) certificate, whereas only 1 1 percent of the control group did— for impacts in the 
cases of both genders that were highly significant. There were smaller, but still statistically 
significant, increases in the proportions of female dropouts in the other services subgroup 
and male dropouts in the OJT USA subgroup who attained a high school credential as a result 
of the program. But there were no significant effects on educational attainment among 
women in the OJT/JSA subgroup or men in the other services subgroup. 

Impacts on Earnings, by Service Strategy Subgroup 

Exhibit S.6 shows the estimated impacts of the incremental services received because of 
access to JTPA on the earnings of adult women and ~ien in each service strategy subgroup . 
As shown in the second column of the top panel i f the exhibit, program impacts on the 
earnings of adult women in the classroom training subgroup followed the expected pattern 
for this type of service: an earnings loss in the first quarter, representing an initial 



Exhibit S.S Impacts ot» Attainment of a Trmmng-Rtkaed High School Diploma or GED 
Certificate: Adult JTPA Assignees Who We,t High School Dropouts, by 
Gender 




Service % attaining HS/OED Impact, m % attaining HS/GED 





training 29.2% 11.356 17.9"* 27.3* 11.3% 

OJT/JSA 9.1 10.9 -1.8 6.4 4.4 

Other mviBM 17.4 9.8 7.6*» 10.2 8.7 

All subgroup* 19.1 10.8 8.2*** 12.7 6.7 

Sample size • I, SIS 



6.0*** 
1,258 



16.0*** 



4.0** 

1.5 




21 
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investment of time in training, followed by a payback period of rising earnings gains in the 
next five quarters, with statistically significant gains of SI 44 and $188 in the last two 
quarters of the follow-up period. The overall 18-month earnings gain of $398 for women 
in this subgroup was not statistically significant. This gain reflected an estimated 8.9 
percent program-induced increase in the hourly earnings of women who worked, which 
more than offset an insignificant -2.5 percent drop in the average number of hours employed 
over the follow-up period (estimates not shown in the exhibit). 

The estimated impacts on the earnings of adult men in the classroom training subgroup 
are less clear. None of the impacts on quarterly earnings was significantly different from 
zero, nor was the overall impact on total earnings over the follow-up period. Moreover, 
the program had no significant impact on the employment rate or hours of work over the 
follow-up period for this subgroup of men (estimates not shown). Thus, there is no evidence 
of a program impact on the earnings and employment of this subgroup. 

In contrast to the pattern for women in the classroom training subgroup, women in the 
OJT/JSA subgroup (middle panel of the exhibit) shov *d an immediate and sustained 
positive impact on average earnings throughout the follow-up period, as might be expected 
with a strategy that emphasizes immediate placement in either an on-the-job training 
position or a regular job. Women in the OJT/JSA subgroup experienced significant 
quarterly earnings impacts of $109 to $144 in five of the six quarters, with an overall gain 
of $742 over the 18-month follow-up period. 

Men in the OJT/JSA subgroup experienced estimated gains of similar magnitude in five 
of the six quarters and over the follow-up period as a whole, although the estimated impacts 
were less often statistically significant. Over the follow-up period as a whole, men in this 
subgroup enjoyed earnings gains of $781. 

Both men and women in the OJT/JSA subgroup experienced a positive impact on hours 
worked; and men, on their employment rate (estimates not shown in the exhibit). Indeed, 
the earnings gains of both women and men in this subgroup were due primarily to increases 
in the number of hours worked, rather than to higher hourly earnings while employed. 

In contrast to the sustained, positive impact on earnings in the OJT/JSA subgroup, the 
program appears to have had only a short-lived effect on the earnings of adult women, and 
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Exhibii S.6 Impacts on Quarterly and 18-Month Earnings: Adult JTPA 
Assignees, by Gender and Service Strategy Subgroup 



Period 


Adult 


women 


Adult 


men 


Control 
mean 

(1) 


impact per 
assignee 

(2) 


Lontrol 
mean 

(3) 


impact per 
assignee 

(4) 






Classroom training subgroup 




Quarter 1 


$ 714 


$ -70* 


$ 1,440 


$ -101 


2 


938 


5 


1,714 


126 


3 


1,066 


52 


1384 


213 


4 


1,189 


79 


2,184 


cn 


5 




144** 


2,171 


151 


0 




188*** 


2387 


-21 


All quartets 




398 


11,780 


418 


Sample size 




2,847 




I JOSS 






OJT/JSA subgroup 




Quarter 1 


$ 1,143 


$ 144*** 


S 1,757 


$ 54 


2 


1^79 


81 


2014 


m *% MP 

135 


3 


1,449 


129** 


2,133 


Io4* 


4 




109* 


2,199 


94 


5 


1,546 


142** 


2,183 


133 


6 


1,570 


138** 


2,169 


201** 


All quarters 


8,607 


742** 


12,456 


781* 


Sample size* 




2287 




2250 






Other services subgroup 




Quarter 1 


$ 960 


$ 39 


$ 1,677 


$ 74 


2 


1,198 


132 


1£51 


104 


3 


1,248 


220** 


2,123 


44 


4 


1,471 


22 


2,199 


44 


5 


1435 


2 


2292 


13 


6 


1,548 


42 


2^74 


-19 


All quartets 


7,960 


457 


12,516 


261 


Sample size 0 




U40 




1.112 



a. Assignees and control group members combined. 

* Statistically fcgnific&ijt ai Ibe .10 tevcJ, ** at ihe .05 level, *** ai the .01 level (two-tailed tcsi). 
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virtually no effect on the earnings of adult men, in the other services subgroup (bottom 
panel). J TP A had a significant impact on women's earnings of $220 in the second quarter , 
followed by smaller, insignificant gains in the later quaxte The estimated impacts on 
hours worked quarterly (not shown) mirrored this pattern—possibly reflecting quicker 
placement in jobs mat were similar to those the female assignees would have eventually 
found without access to JTPA. For men in the other services subgroup, neither the 
estimated impacts on quarterly earnings nor the estimated impacts on hours of work (not 
shown) were statistically significant. 

Overall, then, JTPA led to modest, statistically significant earnings gains for adult 
women in at least one quarter in all three service strategies. The timing of impacts was very 
different across the subgroups, however, and significant for the follow-up period as a whole 
only in the OJT/JSA subgroup. Significant impacts on the earnings of adult men were 
concentrated exclusively in the OJT/JSA subgroup. 

It is important to iterate thai the adults in the three service strategy subgroups differed 
not only in the services they received, but also in their personal characteristics. Our 
forthcoming reports will show mat program intake staff tended to recommend the most job- 
ready applicants for the OJT/JSA service strategy. This difference is evident not only in 
the data on baseline characteristics of the three subgroups (not shown here) but also in the 
earnings of control group members over the follow-up period, shown in columns (1) and 
(3) of Exhibit S.6. These figures indicate that in the absence of program services women 
recommended for OJT/JSA would have earned substantially more than those recommended 
for classroom training and somewhat more than those recommended for outer services. 
Among men the more job-ready applicants tended to be recommended for either OJT/JSA 
or other services; those men recommended for classroom training earned somewhat less 
over the follow-up period than either of the other two subgroups. 

Because of these differences in the three subgroups, one cannot extrapolate the impacts 
for one service strategy subgroup to the women or men served by another. We cannot, Tor 
example, conclude that the program outcomes for adult men in the classroom training 
subgroup would be improved by recommending them for the OJT/JSA service strategy. We 
can only determine which service strategies were effective for those applicants recom- 
mended for them, Whether another service strategy would have been more effective cannot 
be determined on the basis of this study, since we did not observe alternative service 
approaches applied to comparable participant populations. 
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It is also important to bear in mind that the costs, as welt as the impacts, of the three 
service strategies were likely to have varied, as may the longer term impacts. In our final 
report on this study, we wUI present an analysis that compares the costs of Title II-A to its 
impacts over a longer follow-up period. 



Impacts on Earnings, by Ethnicity and Barriers to Employment 

In addition to the three subgroups of women and men based on service strategy 
recommendations, we estimated program impacts on the 18-month earnings of a number 
of other subgroups, defined in terms of personal characteristics measured upon application 
to the program. These estimates help us determine whether the impacts of the program were 
concentrated within certain groups or broadly distributed across all adult women or men. 
In this Executive Summary we present the results for two such subgroups : the major ethnic 
groups and groups facing different barriers to employment.* 

Exhibit S.7 presents the estimated program impacts on the earnings of white, black, and 
Hispanic women (column 3) and men (column 6). Among women the estimated impacts 
differed markedly by ethnic group, with white women showing significant earnings gains 
of $723 over the 18-month follow-up period; black women, an insignificant earnings gain 
of $457; and Hispanic women, an insignificant loss of roughly the same magnitude. 
Moreover, separate tern of the statistical significance of the differences among these impact 
estimates (not shown) indicate the differences among the estimates are likely to represent 
real differences in impact. The estimated impacts for adult men also differed substantially 
by ethnic group, but neither the estimated impacts for individual ethnic groups nor the 
differences in impacts among the subgroups were statistically significant and therefore 
could have arisen by chance. 

In an attempt to narrow the range of possible explanations for these differences in 
impacts among women in different ethnic groups, we estimated adjusted impacts that 
controlled for differences in the distributions of the major ethnic groups across study sites 
and across service strategy subgroups. When we controlled for differences in the 



8. Other key subgroups examined in the report on which this summary is based include those 
defined by work and training histories, public assistance history, household income and composition, 
public housing status, and age (Bloom, On, Cave. Bell, and Doolinle, forthcoming). 
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Exhibit J. 7 /^potf* on the 18-Month Earnings of Major Ethnic Croups: Adult JTPA 
Assignees, by Gender 

Adult women Adult men 

Sample control impaapef sample Control Impact per 
size 0 mean assignee size* mean assignee 

Ethnic group (1) (2) (3) (4) 0) (6) 



White, non-Hispanic 


3,541 


$ 8,007 


$ 723*** 


2,668 


$ 12,929 


$ 625 


Black, non-Hispanic 


1.9S1 


6,829 


457 


1,155 


10,931 


957 


Hispanic 


744 


6,775 


-414 


400 


13,555 


-741 


Full sample* 


6,474 


7,488 


539*** 


4,419 


12306 


550 



a. Assignees and control group moraben combined 

b. Including the three major ethnic groups end American Indiana, Alaskan Natives, Asians, and Pacific Islanders. 
Statistically significant at the .10 level,** at the .05 level, *** at the .01 level (two-tailed te*). 



distributions of the three groups of women across the study sites, the estimated impacts for 
the three ethnic groups were no longer significantly different from one another. This 
finding indicates that the differences in estimated impacts among women in different ethnic 
groups are in part attributable to differences in the sites in which the different ethnic groups 
were concentrated. In particular, Hispanic women were concentrated in those sites which 
had lower than average impacts for women in all ethnic groups. Once we adjust for these 
differences, we see no evidence that impacts differed across ethnic groups. 

To determine whether the effects of the program varied with the degree of labor market 
disadvantage, we estimated impacts on earnings tor subgroups defined by three barriers to 
employment: welfare receipt, limited education, and limited recent work experience. 9 The 
first three pairs of rows of Exhibit S.8 show the estimated impacts on earnings for the 
subgroups of women and men facing each of these barriers, compared with all other women 
and men. 

The mean earnings levels of control group members, shown in columns 2 and 5 of the 
exhibit, illustrate that these barriers were indeed serious obstacles to employment. Control 



9. Welfare receipt is defined as receiving Aid to Families with Dependent Children (AFDC). 
General Assistance, or any other cash welfare benefits upon application to JTPA. Limited education 
is defined as lack of a high school diploma or GED certificate; limited recent work experience is 
defined as having worked less than 13 weeks is Lite year prior to application to JTPA These three 
measures ofbamers to employment are similar to those used in other recent studies of JTPA programs 
(see U.S. General Accounting Office, 1989) 
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EOubit S,8 Impacts on the 18-Month Earnings of Subgroups Facing Selected Barriers to 
Employment; Adult JTPA Assignees, by Gender 







Adult women 






Adult men 




Barrier to 


Sample 


Control 


Impact per 


Sample 


Control 


impact per 


employment 




mean 


assignee 


size" 


mean 


assignee 


(in italic) 


m 


w 


(3) 


(4) 


(5) 


(6) 


Receiving cash welfare 


2,446 


% 5,492 


$ 387 


611 


$ 9,541 


$ -46 


No cash welfare 


3,500 


8.965 


697*** 


3,808 


13,032 


*24 


No high school diploma 












402 


or CED certificate 


1.731 


6,072 


416 


1,249 


10353 


High school diploma 












863* 


or GED certificate 


4316 


8,064 


681*** 


2,873 


13352 


Worked less than 13 














weeks in past 12 mos. 


3,022 


5355 


511" 


2,027 


10308 


207 


Worked 13 weeks or 














more in past 12 mos. 


im 


9,956 


668** 


2392 


14320 


800* 


Number of barriers 














None of the above 


1361 


10,971 


909" 


1360 


15,142 


979* 


One of the above 




7,950 


802** 


1,896 


12,202 


214 


Two of the above 


1,435 


5,756 


379 


772 


8446 


202 


All three of the above 


488 


3,703 


-213 


149 


7,423 


725 


Full sample 


6,474 


7.488 


539*«« 


4,419 


12306 


550 



a. Awjgoeeo sad control group mensbem combined. 

* Statistically tigsi Scant at the .10 level, *♦ at the .05 level, at the .01 level (two-tailed teat). 



group members in all three subgroups facing these barriers earned much less over the 
follow-up period than other women and men. 

Among both women and men the estimated impacts tended to be larger for those not 
facing the barriers in question, although among women the differences in impacts between 
those facing and those not facing a particular barrier were smaller than the differences 
among men. Separate tests for the significance of these differences indicated, however, that 
any differences shown here may have arisen by chance. 

Because some persons who were racing one of these barriers to employment may also 
have been facing one or both of the other barriers, these subgroups overlap to some degree. 
To achieve a clearer distinction among the subgroups in terms of the overall difficulty of 
becoming employed, the bottom panel of Exhibit S.8 categorizes the women and men in 
the sample by the number of these barriers they were facing. Again, the average earnings 
of the control groups indicate that this categorization is strongly predictive of what JTPA 
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assignees would have "anted without the program—control group earnings fall steadily as 
the number of barriers rises. 

For both women and men the impacts were the largest in the subgroup facing none of 
the three barriers. For neither women nor men, however, was the difference in impacts 
among subgroups statistically significant; thus, these differences may merely reflect 
sampling error. 10 

Summary and Comparison with Previous Findings 

Overall, JTPA Title n-A had modest positive impacts on the earnings of adult women over 
the follow-up period; on average, women gained $539 over the 1 8 months following their 
application. The estimated earnings gain for men was similar ($550) but was not 
statistically significant. These overall averages mask substantial variation in both the 
magnitude and time patterns of program impacts among subgroups of women and men, 
however. 

When adult women are categorized by the service strategy recommended by program 
intake staff, the only ones to experience a statistically significant earning impact over the 
follow-up period as a whole were those in the OJT/JSA subgroup, with a gain of $742. 
Women in this subgroup enjoyed consistently positive, statistically significant earnings 
increases of $109 to $144 in five of the six follow-up quarters. Women in the classroom 
training subgroup experienced an earnings loss in the first calendar quarter of the follow- 
up period, followed by growing positive impacts and culminating in significant impacts of 
$144 and $188 in the fifth and sixth quarters. Program impacts on the earnings of women 
in the other services subgroup were significant only in the third quarter, when thes* 1 women 
gained $220 on average; impacts for this subgroup were negligible in subsequent quarters. 

Impacts for adult men were similar in magnitude to those for women, although they were 
less frequently statistically significant. As with the women, only those in the OJT/JSA 
subgroup enjoyed significant earnings gains (of $78 1 ) ove me follow-up period as a whole. 



10. Among the adult female subgroups, for example, there is a 46 ) ercent chance of findisA 
differences at least as large as those shown here even if there were no true differences in impacts among 
subgroups. 
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Estimated impacts on the earnings of men in the classroom training and other services 
subgroups were never statistically significant, either for the follow-up period as a whole 
or for individual quarters. 

These impact estimates are similar in magnitude to those found in the few previous 
evaluations mat have used rigorous experimental designs. For example, studies of state 
work-welfare programs in the early 1980s found impacts in the first two years after random 
assignment that ranged from near zero to about $200 per quarter." Evaluations of 
demonstration programs for displaced workers in Texas and New Jersey found similar 
impacts on earnings in the first year after random assignment. 12 

Comparisons with the results of earlier studies are complicated, however, by the fact 
that the programs involved in those studies provided somewhat different services from 
those in JTPA and served primarily welfare populations and (for men) displaced workers. 
Moreover, the programs for women examined in earlier studies were, unlike JTPA, mostly 
mandatory, and yet had lower rates of participation in employment and training services 
than those of our study sample. 

Finally, when adult women in the National JTPA Study sample are classified by ethnic 
group, significant differences in impacts on earnings emerge, with white women 
experiencing greater gains than minority women, particularly Hispanic women. Further 
tests revealed that these differences in impacts were due in part to the concentration of many 
Hispanic women in study sites with below average impacts for women in all ethnic groups. 
Once this factor is removed, there is no evidence that imj icts differed by ethnic group. 

There were no significant differences in impacts on the earnings of adult men by ethnic 
group. Nor were there any significant differences in impacts on the earnings of subgroups 
of either women or men formed on the basis of several barriers to employment— welfare 
receipt, limited education, and limited recent work experience. This was true both for 
subgroups defined based on individual barriers to employment and subgroups defined based 
on the number of these barriers facing them. 



11. See Gueron and Pauly (1991). 

12. See Bloom (1990) and Corson et al. (1989). 
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Findings for Subgroups of Female and Male Out-of-School Youths 

The study sample of out-of-school youths was classified into the same three service strategy 
subgroups as those used to classify adults: classroom training, OJT/JSA, and other 
services. These subgroups were based on the JTPA services recommended for sample 
members by program intake staff before random assignment. 

Service Strategies Recommended 

The service strategies rpcommended for youths reflect a difference in emphasis between 
JTPA Title II-A programs for youths and those for adults. Programs for adults emphasize 
employment, as evidenced by the fact that program performance standards for adults are 
based largely on job placement rates. In contrast, programs for youths emphasize a broader 
range of outcomes, with performance standards for youths based in part on "positive 
terminations," which include not only job placements but also participation in further 
training and attainment of specific job competencies. 

A comparison of Exhibit S.9 and the earlier Exhibit S.3 indicates that youths were far 
less likely than adults to be recommended for the OJT/JSA strategy, especially if we 
compare female youths with female adults and male youths with male adults. Of the three 
service strategies, OJT/JSA places the greatest emphasis on immediate employment; thus, 
this difference between youths and adults reflects the difference between programs for 
youth and adults in JTPA. In addition, youths were more apt than adults to be recommended 
for the other services strategy, which, as discussed below, had proportionately more youths 
than adults receiving basic education. 

The mix of service strategies recoiunended also differed between female and male 
youths. Female youths were more likely than male youths to be recommended for 
classroom training (44 percent versus 30 percent, respectively) and less likely than male 
youths to be recommended for OJT/JSA (23 percent versus 33 percent). The genders were 
about equally likely to be recommended for other services (33 percent versus 37 percent). 

Enrollment! Rates and Duration, by Service Strategy Subgroup 

Enrollment rates overall were comparable to those for adults, with 66 percent of the female 
youths in the treatment group and 67 percent of the male youth treatment group becoming 
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Exhibit S.9 Service Strategies Recommend?!: 

Out-cf School Youth JTPA Assignees, 
by Gender 
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enrolled in JTPA Title n-A at some time during the 18-month follow-up period. The 
highest enrollment rates were in the classroom training treatment group, in which 72 
percent of the female youths and 75 percent of the male youths were enrolled. The lowest 
enrollment rates were in the OJT/JSA treatment group, with 58 percent of the female and 
59 percent of the male youths enrolled. Other services fell between these two extremes, 
with enrollment rates of 6j percent for female and 68 percent for male youth treatment 
group members. 

Out-of-school youths who enrolled in JTPA stayed in the program for a reisr ' /ely brief 
time. Among female youths the median duration of enrollment v. as 4 months, about a month 
longer than the median for male youths. The service strategy subgroup with the shortest 
enrollments was OJT/JSA, with a median of about 2 months for both females and males; 
the classroom training subgroup had the longest enrollments, at around 5 months for 
females and males. The median for the other services subgroup was about 2 months for 
both target groups. 

Services Received, by Service Strategy Subgroup 

Exhibit S.10 shows that the most common services received by enrollees in each service 
strategy subgroup were those which distinguished it from the others. About 86 percent of 
female youth enrollees and about 80 percent of male youth enrollees recommended for 
classroom training received classroom training in occupational skills, basic education, or 
both. About 85 percent of female and male youths in the OJT/JSA subgroup received on- 
the-job training, job search assistance, or both. And about 80 percent of female youth 
enrollees and 83 percent of the male youth enrollees recommended for the other services 
strategy received basic education, miscellaneous services, or both. 
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Exhibit S. 10 Most Common JTPA Services Received by Treatment Group 
Members Who Were Enrolled in the Program: Out-qf-School 
Youths, by Gender and Service Strategy Subgroup 
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exploration, jab ah*iowing, asd tryoui employment, among other acrvicea. 

The only obvious difference between the types of JTPA services received by youths and 
those received by adult* was in the other services subgroup. Whereas adult enrollees in 
this subgroup received mainly job search assistance and miscellaneous services, youth 
enrollees received mainly basic education and miscellaneous services— further evidence, 
as noted earlier, that JTPA emphasizes immediate employment for adults more than it does 
for youths. 

As with adults, the estimated program impacts on youths reported here reflect the 
differences between the employment and training services received by treatment group 
members, who had access to JTPA, and the services they would have received if they had 
been excluded from the program, as measured by data on control group members. Because 
not all out-of-school youths who were treatment group members actually enrolled in JTPA, 
not all of the youths received JTPA services. Moreover, a number of out-of-school youths 
in the control group received services from non-JTPA providers that were similar to JTPA 
services. Nevertheless, the differences in the services received by treatment and control 
group members were substantial. 
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The differences in service receipt varied by service strategy subgroup. In the classroom 
training subgroup about 48 percent of the female youths and 43 percent of the male youths 
in the treatment group received classroom training in occupational skills, whereas only 3 1 
percent of the female youths and 22 percent of the male youths in the control group received 
this service. In the OJT/JSA subgroup 30 percent of the females and 3 1 percent of the males 
in the treatment group received on-the-job training, while less than 1 percent of both 
females and males in the control group received the service. 

It was not possible to determine the treatment-control group difference in services 
received by youths in the other services subgroup because information on the category of 
services most frequently received by treatment group members, miscellaneous services, 
was not available for control group members. About 29 percent of the females and 35 
percent of the males in the treatment group received a miscellaneous service. 13 

Impacts on Educational Attainment, by Service Strategy Subgroup 

Differences in the services received by youths in the treatment and control groups produced 
differences in the rate at which school dropouts in these groups attained a high school 
diploma or GED certificate. Since half of the youths in the study sample were school 
dropouts, impacts on their educational attainment represent an important result of the 
program. 

As shown in the fourth row of Exhibit S.l 1, among control group members who were 
school dropouts, 17 percent of the female youths and 14 percent of the male youths enrolled 
in an employment and training service aid received a high school diploma or GED 
certificate after random assignment. Among the corresponding treatment group members, 
however, 29 percent of the female youths and 24 percent of the male youths subsequently 
attained a training-related high school credential. The program impact in both cases was 
highly significant. Impacts were also significant for male youths in all three service strategy 
subgroups and for females in classroom training and other services— the two service 
strategies that focused the most on basic education. The impacts were particularly striking 
for females in classroom training. 



13. Many youth treatment and control group members (between 10 and 25 percent) in the other 
services subgroup received basic education or classroom training in occupational skills. But the 
treatment-control group differences in receipt were small for both of these services. 
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Exhibit S.U Impacts on Attainment cfa Training-Related ffigh School Diplotw or GED 
Certificate: OuKfSchooi Youth JTPA Assignees Who Were High School 
Dropouts, by Gender 
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Impacts on Earnings, by Service Strategy Subgroup 

As noted at the outset, the estimated program impacts on the earnings of female youths were 
negligible and those for male youths were substantially negative. Exhibit S.12 provides 
a more detailed look at these impacts for each of the three service strategy subgroups of 
youths during each of the six quarters of the follow-up period. 

In the female classroom training subgroup impacts on earnings were negative and 
statistically significant during the first three follow-up quarters. As with adult women, 
these initial losses probably reflect the earnings forgone by JTPA assignees while they were 
attending classes. Unlike the experience of adult women, however, female youths in 
classroom training did not experience any significant increases in earnings later in the 
follow-up period. Hence, the earnings female youths lost while participating in classroom 
training were not offset by a payback period, at least not by the end of the 1 8-month follow- 
up. 

Female youths in the OJT/JSA subgroup experienced a different pattern. The initial 
impacts on their earnings were moderately positive (and statistically significant in the 
second follcw-up quarter), which may reflect an initial boost in employment produced by 
on-the-job training, job search assistance, or both. But these short-run gains were not 
sustained over time. 
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Exhibit SA2 Impacts on Quarterly and J 8-Month Earnings: Qut-of-School Youth 
JTPA Assignees, by Gender and Service Strategy Subgroup 
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Impacts on the earnings of female youth assignees in the other sendees subgroup were 
negligible in all six follow-up quarters. The mix of predominantly basic education and 
miscellaneous services JTPA provided to this subgroup had no perceptible impact. 

The impact estimates for male youths in the classroom training subgroup were similar 
to those for their female youth counterparts. The impacts on earnings were substantially 
negative and statistically significant during the first follow-up quarter, again, perhaps 
reflecting the costs of being in class instead of employed. And as with the female youths, 
the impacts were negligible for this subgroup throughout the later quarters, suggesting no 
offsetting benefits in terms of increased future earnings. 

Most striking among the findings for male youths were the impact estimates for those 
recommended for OJT/JSA or other services. Estimates for both service strategy 
subgroups were negative in every follow-up quarter, and were substantial and statistically 
significant in many quarters. 

The impact on the earnings of male youth assignees in the OJT/JSA subgroup was 
-$1,313 over the follow-up period as a whole. This statistically significant earnings loss 
represented -10.3 percent of the correspondirg control group's mean earnings. This 
earnings loss was due mainly to a -8.5 percent program-induced reduction in the average 
number of hours worked by assignees during the period; average hourly earnings among 
assignees who worked were largely unaffected by the program (not shown). 

Earnings of male youth assignees in the other services subgroup were reduced by 
-$1,305, or -13.3 percent of what their earnings would have been without access to JTPA. 
This loss was due mainly to a -9.7 percent reduction in the number of hours worked, 
although average hourly earnings when working were also reduced by -4.0 percent. 14 

Overall, then, it appears that the negative program impact on earnings among male 
youths recommended for OJT/JSA or other services reflected mainly the fact that they 
worked fewer hours during the follow-up period than their counterparts in the control group 
did. 



14. The percentage impacts on hours worked and on earnings per hour worked do not sum exactly 
to the percentage impact on total earnings because the relationship between total earnings and its 
components is multiplicative, not additive. 
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These findings for male youths are in sharp contrast to those for adult men, especially 
in the OJT/JSA service strategy. To determine whether these differences were due to age- 
related factors pf r se or differences between the JTPA programs for male youths and adults, 
we tested whether the findings for male youths were an extension of an age-impact trend 
for young male adults or a marked departure from such a trend. The results of this test were 
inconclusive, however, because of the small samples for the age subgroups involved. 



Impacts on Earnings, by Ethnicity and Barriers to Employment 

The estimated impacts on earnings for out-of-schooi youths did not vary systematically with 
the ethnic backgrounds of sample members or with the barriers to employment they faced 
when they applied to JTPA. 

Exhibit S. 13 presents the estimates by ethnic group. The impact estimates for female 
youths did not differ substantially by ethnic group, and no ethnic group experienced a 
statistically significant impact. In addition, separate tests of the statistical significance of 
the differences among the impacts on these groups (not shown) confirm the lack of a 
differential effect of JTPA. 

Among the male youths, there were pronounced differences in the estimated impacts 
among the three ethnic groups. These differences were not statistically significant, 
however, and may therefore simply be due to chance (test not shown). 



Exhibit 5.7 3 Impacts on the 18-Month Earnings of Major Ethnic Grvups: Om-of-School 
Youth JTPA Assignees, by Gender 
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Exhibit SJ4 Impacts an the J 8-Month Earnings of Subgroups Facing Selected Barriers to 
Employment: Out-cf-Schaol Youth JTPA Assignees, by Gender 
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Exhibit S. 14 presents the estimates for subgroups of youths defined in terms of the three 
specific barriers to employment investigated for adults: welfare receipt limited education, 
and limited recent work experience. As was the case fb: adults, these bt, .ers represented 
serious obstacles to employment for youths, as evidenced by the fact that control group 
earnings drop markedly as the number of barriers increases (bottom panel of the exhibit). 

There is little evidence, however, of a systematic relationship between the number or 
nature of these employment barriers and the effect of JTPA on out-of-school youths. 
Among female youths, in particular, there was little difference between the estimated 
program impacts for sample members who faced each of the three employment barriers and 
those who did not. Furthermore, there was no clear pattern in the relationship between the 
estimated program impacts and the number of employment barriers faced. Tests for 
significant differences in impacts among subgroups (not shown) revealed none that were 
statistically significant. 
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Among male youths the differences between the impacts for sample members who faced 
an employment barrier and those who did not appear to be more substantial. For youths 
with limited education or limited recent work experience, JTPA appeared to reduce the 
earnings of males facing these barriers by more than it reduced the earnings of those who 
did not (top panel, column 6). In addition, the more barriers faced, the more JTPA reduced 
future earnings. These differences in impact estimates among the subgroups were not 
statistically significant, however, and so the patterns they imply are only suggestive and 
may in fact be due to chance. Moreover, the difference in impacts was in the opposite 
direction for male youths receiving welfare and those not receiving welfare. 



Summary and Comparison with Previous Findings 

As indicated above, JTPA Title II-A did not appreciably affect the earnings of female 
youths. On average, die program reduced total earnings during the 18-month follow-up 
period by -$182 per female youth assignee (treatment group member), but this estimated 
effect was cot statistically significant. Nor were the impact estimates for female youths in 
the three service strategy subgroups statistically significant. 

The findings for male out-of-school youths were very different. On average, JTPA 
reduced the earnings of this target group by -$854 over the 18-month follow-up period. 
Male youth assignees in the OJT/JSA or other services subgroups experienced large, 
statistically significant earning losses: -$1,313 and -$1,305, respectively. Only the 
estimate for the classroom training subgroup was not statistically significant. 

The large, statistically significant earnings losses experienced by youths recommended 
for the OJT/JSA and other services strategies were due mainly to the fact that treatment 
group members worked fewer hours than they would have without JTPA, not that they were 
paid less for the hours they worked. 

Findings for out-of-school youths in this study are not inconsistent with those from the 
two existing experimental studies of employment and training programs for out-of-school 
youths. 15 The first, the youth component of the National Supported Work Demonstration, 



IS. Although many other employment and training programs for youths have been studied in the 
past, the findings obtained provide little reliable information because of the methodological problems 
endemic to the nonexpenmental research designs that were used, see the review in Betsey, Holbster, 
and Papageorgio, 1985. 
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evaluated an intensive work experience program (Manpower Demonstration Research 
Corporation, 1980): and the second, JOBSTART, evaluated intensive employment and 
training services provided through JTPA (Cave and Doolittle, 1991). The Supported Work 
study found negligible post-program impacts on the earnings of youth participants, who 
were primarily male. JOBSTART found negligible short-term impacts for female youths 
and large negative short-term impacts for male youths, mirroring the findings of the present 
study. 

Both JOBSTART and the youth component of Supported Work targeted seriously 
disadvantaged youths, who make up only a portion of the out-of-school youth population 
targeted by JTPA Title n-A programs. And Supported Work provided far more intensive 
services than are typically available from JTPA. Hence, the three studies of employment 
and training programs for youths focus on different target groups and program services. 

Nevertheless, none of these studies indicates that the programs examined were able to 
improve the earnings prospects of disadvantaged youths; and two of the three studies found 
that such programs actually reduced the earnings of male youths. The experimental 
findings to date therefore are cause for concern. 

Implications of the Findings 

The National JTPA Study is based on only 16 study sites, whiwA are not a probability sample 
of all SDAs and may not be representative of the nation. But to the extent that these results 
apply to other localities, they have important policy implications. 

The study has shown that JTPA is helping to raise the earnings of many of its 
participants, especially among the adults, but it has also identified several groups for whom 
JTPA Title H-A is having no effect or even adverse effects. The program failed to raise 
the earnings of out-of-school youths in general, and it actually reduced the earnings of male 
out-of-school youths recommended for the OJT/JSA and other services strategies. 

Although this analysis has identified those groups not being adequately served by the 
program, it cannot prescribe ways to serve them better. This is because the study was 
designed to observe only the impacts of JTPA as it operated during the study period, not 
alternative ways of serving the same population. 
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Finding ways to improve program performance for those groups negligibly or adversely 
affected by the current program will require experimentation with a range of alternative 
service strategies for these groups and rigorous evaluation of the results. We cannot 
overemphasize the importance of rigorous evaluation of new approaches to serving these 
groups. Experience has demonstrated that simply trying out alternative program strategies 
without rigorous evaluation is not enough. As a National Research Council report 
concluded in a review of more than 60 youth employment and training demonstrations, 
"Despite the magnitude of the resources ostensibly devoted to the objectives of research 
and demonstration, there is little reliable information on the effectiveness of the programs 
in solving youth employment problems" (Betsey, Hollister, and Papageorgiou, 1985). To 
address this deficiency, the authors recommended greater reliance on field experiments 
with random assignment. 

Tiu .°ason it is difficult to evaluate these programs without random assignment is clear 
from the patterns of earnings displayed by the control groups in this study. These show 
that even in the absence of access to JTPA, youths* earnings would have risen rapidly and 
would have varied substantially across service strategies. Without random assignment, it 
is easy to mistake these patterns of outcomes that would have occurred anyway for effects 
of the program. 

Finally, although the current results clearly imply the need for some programmatic 
changes, these results should be regarded as partial. In the final report of the National JTPA 
Study we will extend these findings in several ways. First, we will analyze impacts over 
a longer follow-up period. Growth or decline in the impacts estimated here over the period 
beyond 18 months could materially change the relative impacts among target groups and 
service strategies. Second, we will extend the analysis to include impacts on receipt of 
public benefits from AFDC and food stamps programs. And third, and most important, 
we will compare the impacts of JTPA Title II- A and the three service strategies to their 
costs, to determine the cost-effectivenessof the program at the 16 study sites. 
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